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Perhaps  the  most  significant  contribution  of  the  Naturalistic  Decision 
Making  (NDM)  perspective  is  that  it  has  forced  researchers  to  reevaluate 
the  assumptions  they  hold  about  how  people  make  decisions  in  “real” 
environments  (Orasanu  &  Connolly,  1993).  A  logical  extension  is  to 
examine  the  practical  implications  of  this  theoretical  perspective  on  decision 
making.  In  particular,  it  is  of  interest  to  determine  how  conclusions  drawn 
from  the  NDM  perspective  affect  the  manner  in  which  decision  making 
might  be  optimized,  or  at  least  improved.  Therefore,  the  purpose  of  this 
chapter  is  to  examine  NDM-related  theorizing  from  the  standpoint  of  what 


*The  opinions  expressed  herein  are  those  of  the  authors,  and  do  not  represent  the  official  positions 
of  the  organizations  with  which  they  are  affiliated. 
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it  implies  for  how  we  train  decision  makers.  It  is  based  largely  on  the  panel 
discussion  devoted  to  this  topic;  panelists  included  the  authors,  John 
Schmitt,  Hugh  Wood,  Raanan  Lipshitz,  and  Jon  Fallesen. 

Before  proceeding,  we  have  several  important  points  to  make  regarding 
NDM  and  training,  and  more  specifically,  our  approach  in  this  chapter.  First 
of  all,  it  became  clear  from  the  panel  discussion  that  panelists  were  not 
familiar  with  any  empirical  investigations  of  the  effectiveness  of  NDM-gen- 
erated  training  interventions.  This  conclusion  was  bolstered  in  an  extensive 
literature  search  we  conducted  for  the  purpose  of  identifying  studies  con¬ 
ducted  on  this  topic.  Therefore,  this  chapter  is  necessarily  analytical  in 
nature.  Specifically,  it  focuses  on  generating  propositions  for  training  the 
cognitive  aspects  of  decision  making  as  suggested  by  the  NDM  perspective. 
Furthermore,  we  do  not  attempt  to  review  literature  pertaining  to  this  topic 
(this  has  been  accomplished  nicely  by  Means,  Salas,  Crandall,  &  Jacobs, 
1993;  Orasanu,  1993;  and  others);  instead,  we  seek  to  organize  and  sum¬ 
marize  thoughts  on  this  topic. 

We  also  note  that  the  task  of  generating  cognitive  training  principles 
consistent  with  NDM  is  not  as  straightforward  as  it  may  seem.  In  fact,  the 
nature  of  NDM  itself  provides  a  significant  dilemma  when  it  comes  to 
specifying  training  interventions  because  it  is  in  conflict  with  the  assump¬ 
tions  traditionally  made  about  how  people  make  decisions  in  the  real  world. 
According  to  NDM  theories,  effective  decision  making  is  not  characterized 
by  an  invariant  set  of  steps  or  procedures  as  prescribed  by  normative  theories 
(Orasanu  &  Connolly,  1993).  Instead,  decision  making  is  seen  as  inter¬ 
twined  with  task  accomplishment,  context-specific,  fluid,  flexible,  and  in 
some  respects,  “procedure -free”  (i.e.,  lacking  prescribed  rules  as  suggested 
in  more  classical  views  of  decision  making).  These  are  seemingly  impossible 
goals  to  achieve  through  traditional  training — specifically,  how  do  we  train 
people  to  be  flexible,  or  to  engage  in  decision  making  that  does  not  follow 
a  set  of  predetermined  steps? 

Therefore,  we  have  to  reason  about  decision-making  training,  and 
particularly  about  what  the  goals  of  training  might  be,  with  a  different 
point  of  departure  under  an  NDM  formulation  as  compared  with  classical 
decision-making  perspectives  (Means  et  al.,  1993).  In  fact,  the  most 
fruitful  way  to  characterize  NDM-consistent  training  might  be  to  view  it 
as  a  mechanism  to  support  natural  decision-making  processes,  and  as  a 
means  to  accelerate  proficiency  or  the  development  of  expertise.  The 
terms  “managed  experience,”  “augmented  experience,”  and  “accelerated 
proficiency”  all  come  to  mind  as  descriptive  of  the  training  philosophy 
that  we  believe  is  consistent  with  NDM  theories.  However,  we  also 
contend  that  knowledge,  skills,  and  processes  that  underlie  decision 
making — even  naturalistic  decision  making — can  be  identified  and 
trained.  This  is  not  to  say  that  we  can  (or  should)  specify  a  single  set  of 
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steps  for  decision  making,  only  thatconsistentprocesses  that  supportexpert 
decision  making  can  be  specified  as  a  basis  for  training. 

This  position  implies  that  we  must  be  able  to  describe  wbat  expert 
decision  makers  do — delineate  the  knowledge,  skills,  and  processes  that  may 
underlie  effective  decision  making,  and  also  describe  the  mechanisms  of 
NDM  that  we  seek  to  support.  To  accomplish  this  here,  we  first  use  NDM 
theories  to  establish  a  set  of  characteristics  that  describe  expert  decision 
makers  (based  on  what  are  defined  as  the  characteristics  of  "naturalistic” 
decisions) .  We  then  examine  the  processes,  knowledge,  and  skills  that  NDM 
theories  suggest  are  responsible  for  expert  performance;  in  other  words,  how 
does  the  NDM  perspective  explain  effective  decision  making  performance? 
Next,  we  use  this  information  as  a  basis  to  generate  propositions  regarding 
the  manner  in  which  training  methods,  strategies,  and  content  might  be 
designed  in  order  to  improve  decision  making.  Finally,  we  offer  suggestions 
for  future  research  in  this  area. 


CHARACTERISTICS  OF  EFFECTIVE 
DECISION  MAKERS 

One  way  to  determine  the  implications  of  the  NDM  perspective  for  training 
is  to  address  the  question  of  what  it  suggests  are  the  characteristics  an 
effective  decision  maker.  In  order  to  answer  this  question,  we  first  examine 
what  we  want  trainees  to  be  able  to  do  at  the  conclusion  of  training.  Once 
established,  we  can  use  these  attributes  as  a  basis  to  specify  which  knowl¬ 
edge,  skills,  and  processes  must  be  trained  in  order  to  achieve  targeted 
performance. 

As  noted  elsewhere,  characteristics  of  the  naturalistic  environment  (see 
Orasanu  &  Connolly,  1993)  contribute  to  the  requirements  for  effective 
decision  makers.  On  this  basis  we  hypothesize  that  effective  decision  makers 
can  be  described  as: 

1.  Flexible — Naturalistic  decision  making  contexts  (that  are  ill-struc¬ 
tured,  dynamic,  complex)  require  decision  makers  to  be  flexible.  That  is, 
decision  makers  must  be  able  to  cope  with  environments  that  are  ambiguous, 
rapidly  changing  and  complex.  The  implication  of  this  line  of  thinking  is 
that  expert  decision  makers  have  a  repertoire  of  decision  making  strategies 
that  they  can  draw  on  in  response  to  particular  situational  cues. 

2.  Quick — One  of  the  characteristics  of  naturalistic  decisions  is  time 
pressure  (Orasanu  &.  Connolly,  1993).  This  demands  that  decision  mak¬ 
ers  are  able  to  make  rapid  decisions,  often  in  the  face  of  severe  conse¬ 
quences.  It  is  also  perhaps  the  single  most  important  feature  that 
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ingc^tions^etc!)analVtiCaldeCiSi0nmakin8(i  C  ,^’enerat*n^°^t*0ns,assess' 

nniiv  °fthf  characterstics  noted  bY  Orasanu  and  Con¬ 

nolly  (1993)  are  indicative  of  stress  in  the  decision-making  environment. 

For  example,  ambiguity,  uncertainty,  and  high  stakes  (i.e.,  severe  conse¬ 
quences  for  error),  along  with  other  factors  such  as  high  workload  and 
adverse  physicalconditions,  are  all  stressors  that  can  affect  decision-making 
performance  (Dnskell  &  Salas,  1991).  Therefore,  in  naturalistic  environ 
ments,  expert  decision  makers  must  be  trained  to  operate  under  these 
1993)tl0nS’  Wlth°Ut  Suffenng  Gradations  in  performance  (Means  et  al, 

4.  Adaptive— When  events  unfold  rapidly  in  decision  making,  or 
decisions  involve  multiple  goals,  “static”  models  of  decision  making  do 
not  apply  (Lipshitz,  1993).  Instead,  it  is  clear  that  a  naturalistic  view  of 
decision  makings  suggests  that  decision  makers  must  engage  in  a  contin¬ 
ual  process  of  strategy  assessment  and  modulation.  This  implies  that 
decision  makers  must  recognize  when  and  how  to  apply  a  decision 

proWem  demand^  ^  Change’  °r  modify  that  strategy  in  accordance  with 

5.  Risk  taking— According  to  Orasanu  (1993),  expert  decision  makers 
in  naturalistic  settings  have  a  tendency  to  use  their  knowledge  more 
effectively  to  conduct  nsk  assessment  in  making  decisions.  Obviously,  when 
environments  are  characterized  by  high  stakes,  decision  makers  must-  be 
able  to  assess  the  nsk  associated  with  various  courses  of  action,  and  weigh 
the  consequences  of  error  against  any  potential  payoff.  This  does  not  imply 

at  decision  makers  generate  multiple  options  or  weigh  them  against  each 
other;  only  that  successful  risk  taking  is  often  a  crucial  part  of  making 
decisions  in  naturalistic  settings. 

6.  Accurate— To  say  that  expert  decision  makers  are  accurate  is  stating 
the  obvious  However,  it  is  probably  worth  acknowledging  that,  particularly 
in  light  of  the  fact  that  many  situations  provide  such  high  demand  on  the 

uman  decision  maker,  it  is  perhaps  more  surprising  when  costly  incidents 
and  accidents  do  not  occur  than  when  they  do. 

Now  that  we  have  established  the  characteristics  of  effective  decision 
makers— essentially  delineating  a  set  of  criteria  that  describe  the  “target”  or 
goals  of  teaming— we  can  turn  attention  to  examining  what  NDM-related 
theories  have  to  say  about  how  experts  are  able  to  perform  as  described 
earlier.  The  following  subsections  summarize  what  we  know  about  the 
mechanisms  of  decision  making— in  particular,  how  experts  are  able  to 
perform  in  the  often  difficult,  challenging,  time-compressed  naturalistic 


10.  TRAINING  DECISION  MAKERS 


103 


MECHANISMS  OF  NDM  AND  EXPERTISE 

As  noted,  the  value  of  NDM  theories  in  designing  training  lies  in  what  they 
have  to  offer  regarding  the  knowledge,  skills,  and  processes  that  underlie 
expert  performance.  In  addition,  the  study  of  expertise  has  yielded  insight 
that  has  applicability  to  the  naturalistic  environment.  Given  these  perspec¬ 
tives,  we  now  highlight  the  following  mechanisms  of  expert  decision  making 
that  have  been  studied. 

Situation-Assessment  Skills 

Several  theorists  operating  under  the  NDM  umbrella  have  argued  that 
situation-assessment  skills  are  paramount  in  effective  decision  making  (e.g., 
see  Klein,  1993;  Orasanu  &  Connolly,  1993).  According  to  Lipshitz  (1993), 
all  nine  of  the  NDM-related  theories  he  reviewed  included  an  element  of 
situation  assessment,  with  several  arguing  that  expert  decision  makers  are 
able  to  perform  situation  assessment  more  quickly  and  accurately  than 
novices.  Overall,  it  is  believed  that  this  superiority  in  situation-assessment 
skills  accounts  for  much  of  the  ability  of  experts  to  make  rapid  decisions, 
and  contributes  to  their  decision-making  accuracy.  Two  aspects  of  situation 
assessment  behavior  are: 

1.  Cue  recognition/significance.  For  some  time,  researchers  have  be¬ 
lieved  that  experts  recognize  decision-making  cues  differently  than  novices. 
For  example,  it  has  been  concluded  that  experts  recognize  cues  more  quickly 
and  completely  than  novices,  recognize  patterns  of  cues  better  than  novices, 
and  can  detect  important  features  of  a  stimulus  more  readily  than  novices 
(see  Druckman  &  Bjork,  1991;  Means  et  al.,  1993).  Experts  also  appear  to 
be  better  able  to  frame  decision  problems  so  that  they  can  detect  the 
underlying  structure  of  a  problem  (Orasanu,  1993).  These  skills  all  contrib¬ 
ute  to  the  decision  maker’s  ability  to  perform  effective  situation  assessment. 

2.  Pattern  recognition.  Several  NDM  theories  place  heavy  emphasis  on 
the  ability  of  decision  makers  to  perceive  and  recognize  an  entire  pattern  of 
relevant  cues  in  assessing  a  decision  making  situation  (e.g.,  see  Klein,  1993) . 
Experts  are  also  believed  to  be  able  to  “parse”  the  pattern  of  cues  rapidly, 
ignoring  those  that  are  less  relevant.  According  to  Klein,  it  is  this  recogni- 
tional  process  that  allows  experienced  decision  makers  to  assess  complex 
situations  quickly. 

Organized  Knowledge  Structures 

One  of  the  markers  of  expertise  seems  to  be  the  organization  of  knowledge. 
That  is,  experts  appear  not  only  to  know  more,  but  also  to  organize 
knowledge  more  effectively  (Druckman  &.  Bjork,  1991).  In  terms  specific 
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to  NDM,  experience  allows  experts  to  build  up  knowledge  templates  on 
which  they  can  draw  in  new  decision-making  situations.  This  enables 
decisions  to  be  made  quickly  (because  the  decision  maker  draws  on  preex¬ 
isting  memory  structures)  and  contributes  to  the  accuracy  of  decision 
making.  Details  regarding  template  building  follow. 

Template  Building/Matching.  Several  NDM  theories  suggest  that  experts 
may  store  knowledge  in  templates,  or  as  reference  problems  (Noble,  1993). 
For  example,  Noble  contended  that  these  reference  problems  contain 
objective  features  (i.e.,  the  specific  goals  to  be  achieved  by  the  decision 
maker),  action  features  (i.e.,  a  particular  method  for  accomplishing  the 
objective  specified  by  the  objective  feature),  and  environmental  features 
(i.e.,  criteria  for  adopting  the  solution  method  specified  in  the  reference 
problem).  Although  this  formulation  is  more  detailed  than  others  within 
the  NDM  perspective,  many  converge  on  the  notion  that  over  time,  expert 
decision  makers  build  well-organized  knowledge  structures  that  can  be 
readily  accessed  and  applied  in  decision-making  situations. 


Mental  Simulation 

NDM  theories  do  not  support  the  notion  that  decision  makers  generate  and 
compare  a  series  of  response  options;  instead  they  assume  that  a  recogni- 
tional  process  leads  the  decision  maker  to  generate  a  solution  from  memory. 
However,  when  a  situation  is  novel  (i.e.,  a  template  for  solving  it  does  not 
exist  in  memory) ,  mental  simulation  of  the  potential  solution  is  hypothesized 
to  be  a  primary  mechanism  by  which  a  decision  maker  selects  a  course  of 
action  (Klein,  1993).  According  to  Klein,  this  process  allows  the  decision 
maker  to  determine  whether  the  currently  held  solution  is  viable,  and  leads 
either  to  application  or  adjustment  of  the  solution.  It  contributes  to  the 
accuracy  of  decision  making,  and  helps  decision  makers  to  save  time  in 
crucial  situations. 


Strategy  Selection/Modulation 

As  noted,  flexibility  is  considered  by  many  to  be  an  important  marker  of 
expert  decision  makers.  Several  NDM  theorists  suggest  that,  in  contrast  to 
classical  theories,  no  single  set  of  decision-making  processes  can  be  deline¬ 
ated  that  describes  the  various  types  of  decision  making  that  occur  in  the 
natural  environment  (Lipshitz,  1993).  Therefore,  skilled  decision  makers 
must  learn  to  select  decision-making  strategies  that  are  best  suited  to  the 
nature  of  the  decision-making  situation.  Related  to  this,  the  fluid  nature  of 
many  decision-making  situations  requires  dynamic  regulation  of  decision 
strategy.  That  is,  decision  makers  must  continually  assess  and  modulate  their 
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strategy  in  accordance  with  demands  of  the  decision.  One  way  that  decision 
makers  do  this  is  via  metacognition,  a  skill  that  contributes  to  a  decision 
makers’  flexibility,  and  ability  to  adapt  his  or  her  decision  strategy  when 
required. 

Metacognition.  Expert  decision  makers  appear  to  be  better  able  to  monitor 
their  own  processes  during  decision  making  (e.g.,  Chi,  Bassock,  Lewis, 
Reimann,  &  Glaser,  1989;  also  see  Druckman  &  Bjork,  1991).  For  example 
they  are  superior  to  novices  in  understanding  their  own  level  of  comprehen¬ 
sion,  and  what  is  necessary  to  improve  their  state  of  knowledge.  This 
“executive”  function  is  crucial  in  guiding  a  decision  maker,  particularly  as 
the  problem  changes  and  evolves,  because  it  allows  the  decision  maker  to 
adapt  his  or  her  strategy  as  needed.  Orasanu  (1990)  maintained  that 
effective  metacognitive  skills  also  allow  decision  makers  to  better  manage 
resources  because  they  have  a  more  accurate  picture  of  their  own  strengths 
and  weaknesses,  and  of  the  nature  of  the  problem. 


Reasoning  Skills 

A  host  of  skills  that  we  refer  to  loosely  as  “reasoning  skills”  can  be  hypothe¬ 
sized  to  support  NDM  functions.  In  fact,  the  notion  that  expert  decision 
makers  must  be  flexible  and  adaptable  suggests  that  several  categories  of 
reasoning  skills  may  be  useful  in  expert  decision  making.  For  example, 
Orasanu  (1993)  maintained  that  when  problems  are  ill-defined,  decision 
makers  must  be  able  to  diagnose  situations,  which  requires  them  to  engage 
in  causal  reasoning,  hypothesis  generation,  and  hypothesis  testing.  Orasanu 
also  noted  that  creative  problem  solving  (i.e„  constructing  novel  solutions 
to  a  problem,  or  applying  existing  strategies  in  a  new  or  different  way)  is 
required  when  existing  knowledge  and  procedures  do  not  meet  the  needs 
of  the  current  decision.  Other  reasoning  skills  include  using  analogies, 
critical  thinking  skills  (i.e.,  testing  assumptions,  checking  facts,  seeking 
consistency  among  cues),  and  domain-specific  problem-solving  skills. 

Domain-Specific  Problem-Solving  Skills.  Obviously,  experts  have  richer, 
more  extensive  domain  knowledge  than  do  novices.  However,  it  is  not  only 
the  knowledge  content  that  distinguishes  experts  from  novices,  but  also  the 
manner  in  which  decision  strategies  are  applied  to  that  knowledge  (see 
Hoffman,  1991;  Means  et  al.,  1993).  Specifically,  experts  use  domain 
knowledge  to  determine  when  and  how  to  apply  various  decision  strategies 
(Glaser,  1984).  This  suggests  that  understanding  general  problem-solving 
skills  will  not  improve  decision-making  performance — it  is  only  in  the 
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naturalistic  context  of  a  domain  that  decision-making  strategies  can  aid 
performance. 


IMPLICATIONS  OF  NDM  THEORIES  FOR  TRAINING 

As  was  stated  at  the  outset  of  this  chapter,  we  believe  that  the  overriding 
goal  of  decision-making  training  according  to  NDM  theory  is  to  accelerate 
the  natural  processes  leading  to  expertise.  Now  that  we  have  examined  those 
processes  more  closely,  and  briefly  described  relevant  assumptions  from 
research  on  expertise,  it  is  possible  to  begin  generating  hypotheses  about 
how  NDM-consistent  decision-making  training  might  be  designed.  In  ad¬ 
dressing  this  issue  we  distinguish  two  types  of  propositions  for  training.  The 
first  involves  NDM-derived  suggestions  for  what  should  be  trained— that  is, 
the  content  of  training.  The  second  concerns  implications  from  NDM 
theories  regarding  how  decision  makers  should  be  trained — that  is,  the 
context,  methods,  strategies,  and  media  employed  in  training.  These  cate¬ 
gories  of  implications  are  presented  in  detail  in  the  following  sections. 


Implications  for  What  to  Train  Decision  Makers 

In  considering  what  to  train  decision  makers,  NDM  theories  imply  strongly 
that  context- specific  domain  knowledge  is  a  crucial  aspect  of  expert  decision 
making.  In  addition,  there  are  several  cognitive  processes  and  skills  that 
appear  to  be  required  for  effective  decision  making  (which  are  generated 
directly  from  the  list  of  characteristics  and  mechanisms  delineated  pre¬ 
viously)  that  should  be  trained.  These  include:  (a)  metacognitive  skills  (in¬ 
cluding  the  ability  to  select  decision-making  strategies,  to  modulate 
strategies  as  problems  unfold,  to  engage  in  effective  resource  management, 
and  to  self-assess  and  adjust  as  necessary);  (b)  reasoning  skills  (including 
analogical  reasoning,  causal  reasoning,  creative  problem  solving,  and  critical 
thinking);  (c)  domain-specific  problem-solving  skills  (although  training  “ge¬ 
neric”  problem  solving  strategies  does  not  appear  to  be  fruitful,  training 
decision  makers  in  problem  solving  specific  to  the  domain  may  be  more 
successful);  (d)  mental  simulation  skills  (including  the  ability  to  know  when 
to  apply  mental  simulation  as  a  means  to  evaluate  a  potential  solution);  (e) 
nsfe  assessment  skills  (i.e.,  accurately  assessing  the  risk  associated  with  various 
courses  of  action);  and  (0  situation  assessment  skills  (including  the  ability  to 
make  rapid,  accurate  assessments  of  the  decision  situation  by  improving 
pattern  recognition  skills  and  learning  to  assess  the  significance  of  such 
cues). 

In  addition,  knowledge  organization— fostering  the  organization  of  knowl¬ 
edge  to  support  NDM — is  an  important  training  goal.  Specifically,  the 
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decision  maker  must  be  exposed  to  the  “typical”  situations,  cues,  cue 
patterns,  and  responses  that  characterize  a  domain.  This  process  is  crucial 
as  a  means  to  foster  development  of  necessary  decision-making  templates 
in  memory. 

Implications  of  NDM  Research  for  How  to  Train 
Decision  Makers 

NDM  theories  provide  a  number  of  implications  for  how  decision-making 
training  should  be  conducted.  We  are  particularly  interested  in  proposing 
training  strategies  that  we  believe  can  help  accelerate  the  acquisition  of 
proficiency  or  the  achievement  of  expertise — that  is,  to  aid  decision  makers 
in  learning  and  organizing  domain  knowledge  in  a  manner  that  supports 
complex  decision  making. 

Simulations.  Several  researchers  have  noted  the  importance  of  simulation 
in  training  NDM  skills  (e.g.,  see  Means  et  al.,  1993) .  Simulation  is  a  valuable 
tool  for  training  NDM  skills  because  it  can  accelerate  proficiency  by  expos¬ 
ing  decision  makers  to  the  kinds  of  situations  they  are  likely  to  confront  in 
the  real  world.  Moreover,  simulation  can  be  controlled — the  characteristics 
of  decision  problems,  situational  cues  and  cue  patterns,  and  decision  out¬ 
comes  can  be  provided  as  a  means  to  aid  in  development  of  situation 
awareness,  pattern  recognition,  and  template  building.  For  this  reason, 
scenario/exercise  design  becomes  a  crucial  aspect  of  simulation-based  train¬ 
ing.  In  fact,  the  effectiveness  of  this  type  of  training  will  depend  largely  on 
the  extent  to  which  scenarios  capture  and  display  important  cues  (and  the 
relationship  among  them)  along  with  associated  responses,  so  that  necessary 
templates  can  be  developed.  Simulations  are  also  an  effective  means  to  train 
reasoning  skills,  metacognitive  skills,  and  risk-assessment  skills. 

Guided  Practice  and  Feedback.  Several  studies  have  shown  that  allowing 
trainees  to  practice  on  a  task  without  feedback  may  produce  suboptimal 
decision-making  performance  (Means  et  al.,  1993).  In  the  current  context, 
we  contend  that  it  is  crucial  to  provide  feedback  as  a  means  to  reinforce 
important  “cue  4  strategy”  associations.  In  this  manner,  guided  practice 
(i.e.,  practice  that  incorporates  measures  of  performance  and  specific  feed¬ 
back)  can  be  thought  of  as  managed  experience — providing  decision  makers 
with  examples  that  enable  them  to  characterize  cue  patterns,  build  tem¬ 
plates,  and  associate  effective  responses  to  cues  and  cue  patterns. 

Embedded  Training.  NDM  theories  generally  suggest  that  decision  making 
cannot  be  removed  meaningfully  from  the  context  in  which  it  occurs.  This 
implies  that  training  decision  makers  in  the  environment  in  which  they 
operate  is  recommended  if  possible.  One  way  to  accomplish  this  is  through 
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embedded  or  organic  training.  In  such  cases,  training  is  incorporated  into 
the  operational  system  or  equipment  so  that  training  takes  place  on  the  job 
(with  appropriate  safeguards) .  The  success  of  such  systems  depends  on  their 
ability  to  support  the  development  of  expertise  via  performance  measure¬ 
ment,  specific  feedback,  and  exposure  to  important  decision  cues.  There- 
tore,  scenarios  and  exercises  should  be  developed  that  allow  decision  makers 
to  build  templates  that  represent  a  broad  base  of  problems  they  are  likely  to 
encounter  in  their  actual  decision  environment. 

Cognitive  Apprenticeships.  Recently,  another  on-the-job  training  tech- 
mque  that  has  gained  attention  is  cognitive  apprenticeship  (see  Druckman 
euBjork,  1 99 1) .  Like  traditional  apprenticeships,  which  train  skilled  physical 
performance,  cogmtive  apprenticeships  allow  the  trainee  to  operate  in  his 
or  her  actual  context,  and  require  the  trainee  to  work  closely  with  an  expert 
Briefly,  the  overriding  goal  of  this  type  of  training  is  first  for  the  expert  to 
demonstrate  effective  performance,  and  then  gradually  to  lead  the  student 
through  a  senes  of  constructive  activities  that  allow  him  or  her  to  deepen 
clanty,  integrate,  and  synthesize  knowledge.  As  such,  it  may  be  a  useful 
means  to  train  all  of  the  skills  noted  earlier. 

Multi-Media  Presentation  Formats.  Investigation  into  the  manner  in 
which  know  edge  presentation  formats  affect  knowledge  organization  and 
template  building  is  needed.  Initial  evidence  suggests  that  employing  graph- 
ms  and  animation  may  aid  in  mental  model  development  (e.g.,  White,  1984) 
and  hence,  may  also  have  a  positive  impact  on  NDM  processes.  The  impact 
of  textual,  graphic,  animated,  video,  and  audio  presentation  of  knowledge 
on  decision-making  performance  has  the  potential  to  provide  important 
guidelines  for  training  system  designers. 


CONCLUSIONS  AND  FUTURE  RESEARCH 

In  this  chapter  we  have  attempted  to  provide  ideas  generated  from  an  NDM 
perspective  for  training  decision  makers.  At  the  risk  of  bemoaning  the  point 
our  strong  recommendation  is  that  empirical  assessments  of  NDM-generated 
training  be  conducted.  Until  such  studies  are  completed,  we  will  only  be  able 
to  generate  theoretically  derived  propositions,  as  was  done  here.  In  closing 
we  offer  a  number  of  specific  research  issues  that  we  believe  require  further 
consideration  as  a  basis  to  design  training: 

1.  On  what  basis  do  decision  makers  perceive  similarity  in  situations? 
That  is,  what  triggers  the  activation  of  a  template  in  memory,  or  causes  a 
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decision  maker  to  seek  additional  information  (or  engage  in  mental  simula¬ 
tion)  when  he  or  she  does  not  perceive  the  template  to  match  sufficiently? 

2.  In  what  manner  might  NDM-consistent  training  be  incorporated  into 
more  traditional  training  environments?  For  example,  given  that  decision 
making  is  not  easily  removed  from  its  context,  how  might  NDM  training  be 
integrated  into  other  aspects  of  task  training  in  classroom  situations,  simu¬ 
lations,  on  the  job,  or  any  combination  thereof. 

3.  In  knowledge -rich  environments — that  is,  those  that  require  a  deci¬ 
sion  maker  to  hold  and  access  a  large  volume  of  knowledge  in  making  a 
decision — how  should  expert  knowledge  be  organized  so  that  it  fosters 
access  of  necessary  information  in  decision  making?  That  is,  how  can  we 
initially  present  knowledge  in  a  manner  that  fosters  development  of  tem¬ 
plates  that  are  accessible,  flexible,  complete,  and  useful? 

4.  What  does  the  NDM  perspective  suggest  for  the  manner  in  which  we 
evaluate  decision  making?  For  example,  how  do  we  know  when  someone 
becomes  an  expert?  What  are  the  descriptors  or  criteria  associated  with 
expertise?  What  do  we  mean  by  proficiency?  What  are  appropriate  assess¬ 
ment  strategies  for  decisions  made  under  conditions  that  are  dynamic, 
rapidly  changing,  and  fluid? 

In  summary,  NDM  theories  offer  a  rich  and  exciting  basis  on  which  to 
specify  training  that  will  improve  decision-making  performance.  As  such,  it 
has  the  potential  to  contribute  both  to  the  science  and  practice  of  training 
(Cannon-Bowers,  Tannenbaum,  Salas,  &  Converse,  1991). 
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